Repository navigation
fix(ai): log roast-me stream failures and pass system prompts via system - #35
Merged
Merged
Conversation
…stem` roast-me's streamObject had no onError, so model and schema failures never reached the logs. Grade, take-home and roast-me put the system prompt in `messages`, which triggered the AI SDK prompt-injection warning on every request (~98% of open-silver WARN logs in Dash0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes from the Dash0 log review:
streamObjectkeeps errors inside the stream, so a failed model call or a schema mismatch returned 200 and logged nothing.systemparameter instead ofmessages. This removes the AI SDK prompt-injection warning that made up 1,256 of about 1,280 WARN logs in the last 14 days.streamObject / generateObject / generateText({ - messages: [{ role: "system", content: prompt }, ...rest], + system: prompt, + messages: rest, + onError: ({ error }) => console.error("[roast-me/analyze] stream failed", error), // roast-me only })/api/gradesrc/pages/api/grade.ts,src/resume-checker/prompts/grade.ts(messages()drops the system message and itsparsedarg)/api/analyze-take-homesrc/takehome-checker/index.ts/roast-me/api/analyzesrc/app/roast-me/api/analyze/route.tsroast-mehas the repo's onlystreamText/streamObjectcall.OpenAISdkAIClient(behavioral-checker) is unchanged. It uses the raw OpenAI SDK, which has nosystemoption and doesn't emit this warning.Evidence
The same three requests ran against a local dev server with no AI keys, so every model call fails with an auth error.
bun tscpasses.Merge Danger
Door: two-way
Revert the commit to undo it.
Blast Radius: alerts
The Dash0 rule
exists error in production @ vercel (open-silver)fires on any ERROR log, so real roast-me failures now reach Slack. That is the goal. The prompts are identical, only where they are passed changed, so grading output should stay the same. The happy path was not run with real keys.🤖 Generated with Claude Code